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5. TRAINING EFFECTIVENESS ASSESSMENT: WHERE ARE WE? 

GREG MCGOWAN 


I would like to thank NASA and the FAA for allow- 
ing FlightSafety to participate in this workshop. What I 
hope to do is set a framework for your participation in the 
panel discussion that we will be doing on Thursday. I 
know with all the presentations going on there are a lot of 
questions you will not have the opportunity to ask or get 
answers to. I think the panel discussions will provide an 
opportunity for that kind of participation. 

Concerning the Workshop itself, I look at it from an 
objective standpoint. Even though we are focusing on 
simulators and on certification criteria and so on, I think 
we should be looking at how to provide tools for instruc- 
tors and companies like FlightSafety, to better serve end 
users like Curt Treichel and Jerry Golden, for example, in 
providing safer pilots and safer aircraft operations. 

As an overview to this we will take a look at an 
introduction and historical review, not spending much 
time on the first three or four points. From a historical 
perspective, I think it is important to see where we have 
come from and why we got started in the first place and 
where we are now. Because we are using commercial 
helicopter simulators, we have to ask, how efficient are 
they and how can we optimize their utilization? 

As far as where we are, I think we have to define that 
question in terms of a reference point. We have been beat- 
ing around the bush about this a little bit, but I think this 
Workshop is really concerned with — or at least I am con- 
cerned with — commercial helicopter simulators in the 
United States. I had an opportunity to fly the LHX check 
simulator about 2 months ago. That simulator is a com- 
pletely different animal. It represents some great technol- 
ogy, and interesting things are going to come out of it. 
However, I think the emphasis here must be on commer- 
cial helicopter simulators. We also need to define the 
environment. Are we talking about cost, safety, fidelity, 
and effectiveness of training? I think those are important 
issues that need to be looked at. No one of those issues is 
more important than another; it depends on the end users’ 
requirements, on what is most important to them. I would 


like to take a look at some of those things today briefly, 
and in more detail in the panel discussions. 

From a historical review standpoint, why did we even 
get involved with commercial helicopter simulation? Back 
in the 1970s, Bell Helicopter and Sikorsky Aircraft 
decided to build, for the first time, a commercial heli- 
copter that was not merely a military derivative, the 
Bell 222 and the S-76, respectively. The customers they 
perceived to make up the market for those helicopters 
really consisted of two groups, corporate and offshore, or 
corporate and utility. Certainly there were segments of 
both of those markets that were going to require a simula- 
tor in training the pilots and maintenance technicians for 
those aircraft. And it was the position of both Bell and 
Sikorsky that it would be necessary to have a simulator- 
based training program as part of the overall marketing 
effort for those helicopters. 

That is why the first commercial U.S. helicopter sim- 
ulators were built. You might say the helicopter manufac- 
turers, therefore, are the ones who provided that initial 
impetus to simulator development. But it is really the end 
users, the Curt Treichels and Jerry Goldens, the people 
who use the simulators who drive that market. Without 
that market requirement, the manufacturers would not 
have spent the money on developing simulators. 

Initially, when a simulator-based training program is 
part of a manufacturing agreement, such as we have with 
Bell and Sikorsky, the first course to be developed is ini- 
tial training, which is then quickly followed by recurrent 
training. 

I am proud to say that we are now getting into what I 
call generic training, using simulators that are designed 
for specific aircraft, but using them in a generic way. For 
example, there are the Emergency Medical Service (EMS) 
helicopter pilot recurrent course and the instrument 
refresher courses. We have pilots flying Augustas and 
small Bell products, as well as the Aerospatiale products, 
which don’t have simulators, enrolled in courses in which 
they are using an S-76 or Bell 222 simulator to get as 
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much as they can out of a simulator-based training pro- 
gram. They are practicing things like crew coordination, 
cockpit management, and instrument procedures. The 
technology is developed to the point that we can duplicate 
the actual aircraft, but we tend to forget the other applica- 
tions that we used years ago in the Links and Dehenel 
trainers and the training devices, which are still applicable 
in the current generation of simulators. 

We are really only talking about three simulators. We 
have two aircraft for which there are certified simulators, 
those being the 222 and the S-76B. There is also a third 
training device out there that did some ground breaking 
on its own from an exemption standpoint, and that is the 
S-76A. 

More accurately, the S-76A is for all practical pur- 
poses a training device. It is the most sophisticated train- 
ing device I have ever seen. 

At the end of 1990, there were 174 Bell 222s, and 
319 S-76 aircraft worldwide. A total of 3,747 pitots were 
trained in the Bell 222 and 5,096 were trained in the S-76; 
that is, in all types of training between 1980 and 1990. 

The check ride numbers are 354 for the Bell and 2,333 for 
the S-76. The reason I point this out is because there are 
significant opportunities for data collection here. There- 
fore, these two pilot training devices were used to train 
almost 9,000 pilots and to give about 2,700 FAR checks. 

A breakdown of those check rides shows virtually all of 
the 61.57 instrument competency checks (1,296) were 
done in the Bell 222 simulator. There are reasons for that I 
don’t need to go into, but the primary one is that the 
61.58 PIC check is not required in the Bell 222; as a 
result, the best thing you can do is a biennial flight review 
or instrument competency check. 

The 61.57 instrument competency check totals 
(1,296) are from a combination of the Bell 222 and the 
S-76. The low numbers of 135.293 (129) and 135.297 
(121) checks are a result of our doing them for only a 
couple of years. 

Regarding the commercial helicopter simulators — 
without going into a lot of detail, I certainly will provide 
syllabuses for any of the courses to anyone who wants 
them; just give me a call and we will mail them out. 

The initial training course Is Z weeks long. It was cer- 
tainly the first course developed for either of the S-76 sim- 
ulators, or for the Bell 222, for that matter. Most of the 
recurrent training courses are 4 days long. We do have 
specialized courses of 3 and 5 days for certain operators 
and special requirements. One of the points I want to 
make here, though, is that before we had our first exemp- 


tion, our generic courses, things like the recurrent training 
and the initial training we were doing, were well attended, 
even though the pilots were getting absolutely no credit 
whatsoever. I think that that is an important point for all 
of us to remember: the end user, the pilot, the operator, 
the company, recognized the value of the training, and 
they were willing to pay for it in many cases without any 
checking credit, without any training credit whatsoever. 
On the other hand, I think we also need to realize that just 
because they have been doing it does not mean they are 
going to continue to do it, especially as costs go up. 

figure 1 shows what we call a pilot proficiency 
record. Actually, it is a five-page document. This is what 
our instructors use to evaluate pilots undergoing training 
and checking at the Center. It is a part of the pilot's train- 
ing record. The shaded items are those that would be 
required for an ATP check or for a pilot command 
61.58 proficiency check, I believe the regulation reads 
that the same Items and maneuvers that would be done For 
the initial issuance of type-rating would be required or 
recommended for 61.58 pilot proficiency check. 

The unshaded Items are those things pilots are 
required to complete during our course of instruction, 
which, by the way, is FA A approved. They also receive 
what is called a flight-safety proficiency card. It has been 
mentioned that we did so much more than required. For 
example, on engine malfunction, the high-side governor 
failure was mentioned. We have them do high-side and 
low-side governor failures. They cannot do those in the 
aircraft, and it is something pilots make mistakes on. They 
can get that experience only on the simulator. That is what 
simulator-based training is all about. We can talk about 
this more in the panel discussion, if we get a chance. 

A little history of the exemptions might be in order. 
Exemption 4609 was issued in January 1986 (table 1). I 
think we started the request in early 1984, 1 think we first 
had a meeting up in Washington, D.C. It took time, 
because we were breaking new ground; but we eventually 
got it for the S-76 training device and for the Bell 222 
simulator, with which we do the PIC check and flight 
review. Numerous prerequisites and recency-of- 
experience requirements are stated. 

In almost all cases, even with fixed-wing simulators 
in which checking or training are done, an approved 
course of instruction is included. You don’t just go out 
and use these simulators to do a check ride. There is an 
approved program of instruction; the same is true for this 
exemption. For example, aeronautical experience from 
( ,61 requires 50 hours in the last 12 months, 5 hours 
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FLIGHT SAFETY 
Pilot Training Record 

Captain 


Organization. 
Course 


GRADING LEGEND 

1 = Proficient 

2 = Normal Progress 

3 = Additional Training Required 

4 = Unsatisfactory 

D - Discussed/Demo 

ltem(s} graded 3 or 4 must be 
defined under remarks 




MANEUVERS AND PROCEDURES 




Figure 1. FlightSafety pilot proficiency record. 
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Table 1. Exemption 4609 


Table 2. Exemption 5067 


1, Exemption issued 28 January 1986 

2. Applicable to S-76 training device; Bell 222 simulator 

3. Prerequisites/recency requirements: 

Approved training course 
Aeronautical experience (61.161) 

50 hours preceding 12 months in type 
5 hours PIC last 60 days, make and model 
3 takeoff and landings last 90 days 

4, Amended 23 June 1988 to include S-76B level C 

simulator 


PIC, and three takeoff and landings in the last 90 days. 

The customer base we are addressing has no problem 
meeting these. It was amended in June to include the 
S-76 simulator. I am using those terms loosely because 
they don’t really apply, We cannot call it a level C; it is an 
approved helicopter simulator. That is the proper termi- 
nology, but if you use it people ask you so many questions 
it is better to call it a level C and not have to explain all 
this. 

Exemption 5067 was issued 29 June 1989; it is 
applicable to level C simulators. It is an outgrowth of the 
approval we got with the simulator, and it is approved for 
conducting the checks shown in table 2. Those pilots 
undergoing these checks have to certify that they have, for 
example, done three slope takeoffs and landing within the 
last 90 days. This is not a real big problem when you 
consider that runways are usually crowned and therefore 
have some degree of slope. The other prerequisites 
include 100 hours in the preceding 12 months, 10 hours in 
the S-76, 50 hours in the preceding 6 months, visual 
inspection, 360° pedal turn in hover, normal takeoff from 
hover, manual flown precision approach, and steep 
approach and landing. 

As soon as an exemption or regulation requires that a 
pilot do anything in an aircraft, with respect to checking 
or training, you will eliminate a certain segment of that 
population that would otherwise train in the simulator. 
They won’t train in the simulator because it costs you 
about $2,500 an hour to fly the aircraft. And it can cost 
even more if travel is involved in getting to the examiner. 
So a lot of these decisions are based very much on eco- 
nomics. That’s something that we need to talk about in the 
panel discussion. 

A question that really needs to be asked is how effec- 
tive are commercial simulators? Objectively, I think more 
research is needed. That is one reason I showed you the 


1. Exemption issued 29 June 1989 

2. Applicable to S-76B level C simulator 

3. Approved to conduct the following; 

6 1 .56; 24-month flight review 
61.57: Day/night landing currency 

61.58: 12/24-month PIC check 

61.163: ATP rotorcraft (90%) 

i 35.293: Recurrent testing 
1 35.297: Instrument proficiency 

4. Prerequisites/requirements: 

Approved training course 

Three slope takeoffs/landings 90 days 
100 hours preceding 12 months (10 hr S-76) 

50 hours preceding 6 months (5 hr S-76) 

61.163 ATP/add-on, flight test in S-76 
Visual inspection 
360° pedal turn in hover 
Normal takeoff from hover 
Manually flown precision approach 
^^^^tee^ggroac^n^andmg^h^go^^^^ 

numbers that we have. The people are coming to train, 
and as a result the opportunities for collecting data are 
there. At FlightSafety we certainly are not experts at col- 
lecting data. I don’t know what kinds of questions to ask 
these people or what kinds of maneuvers to ask them to 
see and duplicate.' 

There is one thing I want to mention when talking 
about duplication. When we are evaluating these simula- 
tors and we go out and fly the aircraft and we come in and 
fly the simulator, we need to fly that helicopter at night. 
We need to be doing those 360° pedal turns in a hover at 
night over a runway similar to what we have in the air- 
craft, or in the simulator. T realize in some cases we are 
looking at breakout forces and things that don’t really 
make a difference visually. But when you are subjectively 
evaluating the overall quality of a simulator I think ii is 
unfair to go out in the daytime with all the daytime visual 
cues and compare it with a night visual system. 

Subjectively though, I think the simulators are very 
good for a number of reasons. We have the data, we have 
the pilots, and we have a lot of FAA pilots that have gone 
through training who can tell you about the level of 
instruction, the kinds of things that can be simulated, the 
maneuvers that they can do in the simulator and then 
compare with the actual aircraft. We have some people 
say the simulator doesn’t hover right, and we have others 
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who say it hovers just like the aircraft. That is why we 
need to collect more data and find out what the weak- 
nesses and strengths are. 

We also need to keep in perspective the overall idea 
that there is a lot more positive to be said about the simu- 
lator than negative. The article I mentioned earlier about 
the helicopter that went down in the river off of the Wall 
Street heliport is a good example. This is a quote from the 
pilot, Sandy Kaplan. “The engine quit on departure. We 
didn’t have enough power to continue. We just went 
down, just like we practiced at FlightSafety — you bet!” 
That is an example of the benefits they gained from train- 
ing received in a simulator that they could not have 
received in the aircraft. 

Lastly, how are we going to optimize the effective 
utilization of helicopter simulators? We already talked 
about some of them. I think we need to look at the regula- 
tions and to have an opportunity for giving the two differ- 
ent types of check rides so you can substitute things that 
can be done in the simulator for things that perhaps cannot 
be done in the aircraft. In other words, maybe one low- 
side governor failure and one high-side governor failure 
and an engine fire could equal one 360° pedal turn in a 
hover — for lack of a better example. We need to look at 
the philosophy of simulator use. 

That includes looking at things such as I just men- 
tioned. We need to do a better job of training our instruc- 
tors. We have problems as a company, as a simulator 
trainer company that uses instructors for simulator train- 
ing. We need to better educate those instructors, we need 
to do a better job of training them in cockpit resources 
management, in how to do a better job of debriefing to get 
as much as we can out of the training tools. I refer also to 
cost. For example, Jerry Golden and Curt Treichel — they 
are the one who ultimately decide whether they will use 
the $ 10-million and $ 12-million simulators that we train 
with. 

MR. McDANIEL: By the way, I flew that approach 
to Wall Street and landed in the water as well. I did it in 
his simulator a couple of weeks ago. We practice doing 
those things and we did it successfully the first time we 
tried it in the simulator. And after going through the pro- 
cedures with instruction, we did high-side governor fail- 
ure, we did low-side, tail-rotor failures, fixed pitch, all of 
those things. Quite frankly we were not always successful 
on our first attempts on those things in the simulator. But 
anyhow, the thing is, there is some excellent instruction 
out there that is available with this kind of thing. As we 
said, we had a number of discussions but active conversa- 


tions on the usefulness of it, and I am convinced that it is 
a very useful training instrument and something that we 
need to get credit for and bring into the system. That is 
really why we are here. 

MR. CARVER: Just three observations on that very 
excellent rundown. There is a lot of thought in what you 
said. 

First of all, as far as training and checking are con- 
cerned, everybody wants credits for training devices or 
simulators or whatever. Of course the observation of pilot 
regulators is that pilots need more training than that which 
a regulator requires, so as long as training is not negative, 
then most regulators would support what you have just 
suggested, that is, without necessarily having credits, 
because it is the commercial public transport company 
that is responsible for the pilot training, and what the reg- 
ulator wants is really a snapshot of something at the end. 

As far as effectiveness is concerned, there are one or 
two other points. Effectiveness depends on the fidelity of 
the simulator, on its maintenance records above all. There 
is a thought there with regard to the complexity of the 
device and what effort the company is willing to put into 
its maintenance, and the ability and imagination of the 
instructor-examiners. I definitely agree there with you. 

And finally, I am not a rotary pilot, but as far as the 
simulator is concerned, rotary really requires more pilot- 
ing skills, so I think we have to be careful when giving 
licensing credits to a simulator. But certainly the generic, 
the human factor is certainly an area in which it is useful. 

MR. McGOWAN: Those are good points. I hope you 
come to the panel discussion because those are the kinds 
of things I think we need to talk about. That is the whole 
ideas of this presentation: to whet your appetite for that 
panel discussion. 

MR. LOMBARDO: When I first went to work for 
FlightSafety back in 1979 and 1980 in the King Air pro- 
gram, one of the things that I was very dismayed to dis- 
cover was that the training for the instructor was very 
minimal, and there was an assumption, which it appears 
will continue through the 1990s, that if you are a good 
pilot you must be a good instructor. And what Curt will 
testify to is we did the job with the Blue Box and we can 
do a better job with more sophisticated equipment, but 
what industry needs, and I have had a devil of a time try- 
ing to convince anybody of, is guidelines for a structured 
training program for people who are going to instruct in 
simulators. 

Typically what happens is we find somebody who is 
typed in the aircraft or has experience in the aircraft and 
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we put him in the box and assume he knows how to teach 
in a simulator. These people tend to fall into one of two 
categories: (1) those who use the simulator exactly as they 
use the aircraft, in which case they underutilize the 
equipment; or (2) those whose approach is let's see what I 
can do to them today, who overload the students. I am not 
a helicopter pilot; I am a fixed-wing pilot. Still, I would 
say that what needs to be done in the helicopter industry is 
to develop the guidelines for, or formulate a committee to 
put together, a program to teach people how to teach in 
simulators. You can do more with a good instructor and 
less accurate piece of hardware than you can do with a 
highly accurate piece of hardware and a poor instructor. 

MR. McGOWAN: I agree with that last point that 
you made, totally. I will say that more than 4 years ago 
FlightSafety finally recognized part of what you said and 
developed an instructor development course that all of our 
instructors now go through. It is a 5-day course, standard- 
ized, taught in one location in Texas, and all of our 
instructors have to go through it. 

There is a recurrent instructor course. It is not a do-all 
and end-all for the problem you are talking about. The 
Center is also ultimately responsible, through standard- 
ization, to ensure that the instructor is using these tools 
effectively. The FAA also has a part in that. Once you 
become a pilot-proficiency examiner you have to undergo 
check rides and they actually sit in on the check ride or a 
portion of it. A lot of the checks we do are progressive 
checks, and they have an opportunity to criticize or make 
comments on how you are doing your job, whether you 
are doing it effectively or not. These are important things. 
We could have a whole workshop dedicated to the subject 
of instructor training. 

MR. McDANIEL: I agree with your point that a 
good pilot does not necessarily make a good instructor. I 


have known many very good pilots who are not very good 
at instructing. I would say that a good instructor pilot 
probably does have the skills to be a good simulator 
instructor. But there are differences between instructing in 
the actual aircraft and in the simulator and some strengths 
of the simulator, some capabilities of the simulator, make 
instructing in the simulator different from instructing in 
the real aircraft. I think we all appreciate and recognize 
this. I agree, you do need some kind of instructional pro- 
gram for the simulator instructor so he can best take 
advantage of the strengths of simulator use. 

MR. CLENNEY: I agree 100%, because I also have 
been an instrument flight examiner in both airplanes and 
helicopters. When you start giving an instrument flight 
examination in a simulator, you are also now air-traffic 
control, and you have to plan your air-traffic control so it 
will be realistic for your pilot, the pilot is busy, but the 
instructor is busier. So I highly endorse this idea. 

MR. McGOWAN: You are absolutely right. That is 
one of the things that have to be done in your instructor 
training. Probably the most difficult thing to teach an 
instructor is how to think further ahead than he or she has 
ever thought before because you have to be the ATC func- 
tion, you have to be the Center, and you also have to, in 
some cases (for example in the EMS recurrent course) 
play the role of doctor, nurse, or EMT in the back of the 
helicopter during a loft scenario. It is a really busy job and 
it is actually, from a planning standpoint, much easier to 
do in the aircraft, because then you are really at the mercy 
of the system. You either get the ILS approach or you 
don't. In a simulator you have to plan for it. If you haven’t 
done the proper planning, in the simulator there is no sys- 
tem to take care of you. 
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